Mean-Variance Optimization in Markov Decision Processes
نویسندگان
چکیده
We consider finite horizon Markov decision processes under performance measures that involve both the mean and the variance of the cumulative reward. We show that either randomized or history-based policies can improve performance. We prove that the complexity of computing a policy that maximizes the mean reward under a variance constraint is NP-hard for some cases, and strongly NP-hard for others. We finally offer pseudopolynomial exact and approximation algorithms. keywords: Markov processes; dynamic programming; control; complexity theory.
منابع مشابه
Algorithmic aspects of mean-variance optimization in Markov decision processes
We consider finite horizon Markov decision processes under performance measures that involve both the mean and the variance of the cumulative reward. We show that either randomized or history-based policies can improve performance. We prove that the complexity of computing a policy that maximizes the mean reward under a variance constraint is NP-hard for some cases, and strongly NP-hard for oth...
متن کاملRisk-Sensitive and Mean Variance Optimality in Markov Decision Processes
In this note, we compare two approaches for handling risk-variability features arising in discrete-time Markov decision processes: models with exponential utility functions and mean variance optimality models. Computational approaches for finding optimal decision with respect to the optimality criteria mentioned above are presented and analytical results showing connections between the above op...
متن کاملRisk Aversion in Markov Decision Processes via Near Optimal Chernoff Bounds
The expected return is a widely used objective in decision making under uncertainty. Many algorithms, such as value iteration, have been proposed to optimize it. In risk-aware settings, however, the expected return is often not an appropriate objective to optimize. We propose a new optimization objective for risk-aware planning and show that it has desirable theoretical properties. We also draw...
متن کاملAccelerated decomposition techniques for large discounted Markov decision processes
Many hierarchical techniques to solve large Markov decision processes (MDPs) are based on the partition of the state space into strongly connected components (SCCs) that can be classified into some levels. In each level, smaller problems named restricted MDPs are solved, and then these partial solutions are combined to obtain the global solution. In this paper, we first propose a novel algorith...
متن کاملA mixed Bayesian/Frequentist approach in sample size determination problem for clinical trials
In this paper we introduce a stochastic optimization method based ona mixed Bayesian/frequentist approach to a sample size determinationproblem in a clinical trial. The data are assumed to come from a nor-mal distribution for which both the mean and the variance are unknown.In contrast to the usual Bayesian decision theoretic methodology, whichassumes a single decision maker, our method recogni...
متن کامل